Appendix D — Assignment D

Instructions

You may talk to a friend, discuss the questions and potential directions for solving them. However, you need to write your own solutions and code separately, and not as a group activity.
Make R code chunks to insert code and type your answer outside the code chunks, in the template provided. Ensure that the solution is written neatly enough to understand and grade.
Quarto-render the file as HTML to submit. For theoretical questions, you can either type the answer within the template and include the solutions in this file, or write the solution on paper, scan and submit separately.
The assignment is worth 100 points, and is due on 9th June 2025 at 11:59 pm.
Use a significance level \(\alpha = 5\%\) in all hypothesis tests, unless otherwise specified.
Five points are properly formatting the assignment. The breakdown is as follows:

There aren’t excessively long outputs of extraneous information (e.g. no printouts of entire data frames without good reason, there aren’t long printouts of which iteration a loop is on, there aren’t long sections of commented-out code, etc.). There is no piece of unnecessary / redundant code, and no unnecessary / redundant text (2 pt)
Final answers of each question are written clearly (1 pt).
The proofs are legible, and clearly written with reasoning provided for every step. They are easy to follow and understand (2 pt)

10 points will be deducted in case the provided template is not used (for coding / text-answer questions), and/or the template is not rendered using Quarto markdown.
For questions involving derivations (Q2a-c, Q3 in this assignment), you are allowed to do them on paper, scan and upload separately. However, you are welcome to type the derivations in this template.
To maintain family-wise error-rate, use Tukey’s correction, wherever needed. Don’t use Bonferroni’s correction. This is for consistency and simplicity of solutions.

1) LSD (without replicates)

This is the same problem as in Assignment C, but the designs will be different.

During the Covid-19 pandemic, several schools changed their mode of instruction from in-person to virtual or hybrid. As a result, educators decided to test the effect of virtual / hybrid / in-person education on the ACT scores of high school students.

Based on exploratory research, it was found that the income status of a student is likely to effect the ACT score, and it may also interact with the teaching mode and school in effecting the ACT score. To analyze these effects, the average ACT score was collected for students of a given income status (low income / medium income / high income), school, and being taught with a particular teaching mode.

1a) Design

Develop a latin square design to conduct the experiment. Draw a schematic diagram of the design.

(2 points)

1b) Identify LSD

Two designs were proposed to analyze student scores. The files act_scores4.csv and act_scores5.csv consist of the results of the experiments conducted with the respective design.

Which of the two designs is a latin square design? Justify your answer.

(4 points)

1c) Comparison (expected)

What is the ratio of the expected width of confidence interval of the difference in the estimates of any pair of teaching mode effect in case of the design identified in 1(b), and the design in Q2 (RCBD without replicates) of assignment C? Assume that the expected variance of the error estimate is the same in both the cases.

(4 points)

1d) Comparison (actual)

Calculate the ratio of the actual width of confidence interval of the difference in the estimates of any pair of teaching mode effect in case of the design identified in 1(b), and the design in Q2 (RCBD without replicates) of assignment C.

(4 points)

1e) Justifying comparison

Which design provides a narrower confidence interval as per the results of Q1(d)? Do the results of Q1(d) contradict the results of Q1(c)? Why or why not?

(4 points)

2) LSD (with replicates)

To increase the precision of the results, 3 replicates of the latin square design identifed in 1(b) are considered. Assume the schools are the same in all the 3 replicates.

2a) Two-factor interactions

How many two-factor interactions may exist? Note that the replicate is also a factor having a fixed effect.

(2 points)

2b) Estimable two-factor interactions

Which of the two-factor interactions can be estimated, and which of them cannot be estimated? Write the unbiased estimates of the two-factor interactions that can be estimated.

For the two-factor interactions that cannot be estimated, show that they are confounded with other two-factor interactions, and thus their effect cannot be separately estimated.

Your solutions must have the model equation and all the relevant model aggregates.

(18 points = 6 points for writing the model equation & the relevant aggregates + 6 points for identifying the confounded 2-factor interactions + 6 points for writing the estimates of the estimable 2-factor interaction effects)

2c) Expectations

The data act_scores6.csv consists of the results of the experiments based on the replicated latin square design. Assume the model consists of the main effects, including the effects of the replicates, and the estimable interaction effects identified in Q2b

Derive the expressions for the:

Expected mean teaching-mode sum of squares [\(E(MS_{TM})\)],
Expected mean school sum of squares [\(E(MS_{School})\)],
Expected mean income sum of squares [\(E(MS_{income})\)],
Expected mean interaction sum of squares for each of the estimable interaction effects identified in Q2b.

As the derivations will be similar, you are allowed to skip steps, and write expressions directly wherever you can.

(6 points)

2d) Variance of difference

Find the variance of the estimate of the difference in any pair of teaching mode effects.

(4 points)

2e) Confidence interval

Use the result of the previous question to obtain the width of the confidence interval of the difference in the estimates of any pair of teaching mode effects.

(2 points)

2f) Comparison

Has the width increased or decreased as compared to that in 4(h) of assignment C? Why?

(4 points)

2g) Best teaching method based on income

Which is the best teaching method based on the income status of the student, i.e., which is the best teaching method for:

Hign income students,
Medium income students,
Low income students.

Include uncertainties into account, i.e., there may be multiple best methods for a given income status.

(3 \(\times\) 2 = 6 points)

2h) Interaction in LSD

As shown in 2(b), some two-factor interactions could not be estimated with the given replicated latin square design. Propose another experimental design, which is a combination of 3 latin square designs, such that all those 2-factor interactions could be estimated. Is there another name for the proposed design?

(4 + 2 = 6 points)

3) Computer models

Derive the uncertainty in the estimate of the metamodel in case of ordinary kriging. Compare it with the uncertainty in the estimate of the metamodel in case of simple kriging. Is it bigger or smaller? Why?

(7 + 1 + 2 = 10 points)

4) Design of computer experiments

Suppose the computer model is given by the borehole function. Develop an ordinary-kriging metamodel, and a universal-kriging metamodel for each of the following 10-run designs:

Maximin design
Maximin LHD
MaxPro LHD
MaxPro design

Compute the RMSE (root mean squared error) of each of the 8 metamodels on a 1000-run sobol sequence.

Compare the RMSEs, and comment about the relative performance of the designs and the metamodels.

Which of the model-design combination has the least uncertainty in its predictions, on an average?

(8 x 1.5 points for the RMSEs + 4 points for uncertainty + 3 points for comments = 19 points)